It is well known in the art to control the quality of the printing of banknotes and other similar documents.
In the prior art, when checking the printing quality on paper and especially the printing quality of paper securities, electronic automatic inspection means are used which comprise one or more black-and-white or color cameras to capture the images to be inspected. These images consist of matrices, usually rectangular matrices, comprising pixel values which are representative of the quantity of light reflected by the inspected material (in reflective inspection where one side of the inspected material is checked) or transmitted through the inspected material (in transparency inspection where the transmission properties of the inspected material are checked). In other words, the image is subdivided into a plurality of pixels each having a densitometric value representative of the light reflected or transmitted by a corresponding local region of the inspected material. The number of pixels relating to an image is a function of the resolution of the camera. In a monochrome (black and white) system, the image is described by a single matrix, while in polychrome systems the image is usually described by as many matrices as there are chromatic channels used. Normally, for the RGB (Red, Green, Blue) type, three chromatic channels are used. The procedures used to carry out this type of automatic check are based on the following schemes:
From a set of sheets regarded as being acceptable, a model of acceptable printing quality is constructed. Various techniques are used to construct this model. For example, from the set of sheets regarded as being acceptable, an average image is calculated, that is to say an image which is described by a matrix in which each pixel is associated with the average value obtained in the set of test sheets.
Another procedure associates each pixel with two values, one is the minimum values which has been attained in the set of test sheets and the other is the maximum values. Thus, for each image, two matrices are used, one with the minimum value and the other with the maximum value. Of course, if the image is a polychrome image, two matrices per colour channel are obtained.
The above procedures are in particular disclosed in European patent applications EP 0 527 285 and EP 0 540 833 (corresponding to U.S. Pat. No. 5,317,390 and U.S. Pat. No. 5,384,859).
When producing the images to be inspected, each pixel of the image to be inspected is compared with the pixel of the model thus obtained. If the difference exceeds a predetermined threshold value or if it lies outside the minimum-to-maximum range, the pixel is regarded as having a printing defect. In the end, the total number of defective pixels determines whether or not the image will be rejected.
More precisely, according to U.S. Pat. No. 5,317,390, the procedure for judging the quality of a printed image printed on a printing carrier comprises: dividing a printed image to be judged into a multiplicity of image pixel elements of a preselected size; determining a nominal ink density value for each of said image pixel elements and storing such nominal ink density values in a reference image storing device; determining from a multiplicity of printed proof images judged to be acceptable the actual proof ink density values for each of said image pixel elements: obtaining from said actual values a maximal acceptable ink density value (FD MAX) and a minimal acceptable ink density value (FD MIN) for each of said image pixel elements to provide an ink density tolerance range for each said image pixel elements; allocating to said stored nominal ink density value for each of said image pixel elements an error tolerance range corresponding to the maximal and minimal acceptable ink density values FD MAX and FD MIN; measuring the actual ink density value for each image pixel element for a printed image to be judged; and comparing said measured ink density values for each image pixel element to said stored nominal ink density value and error tolerance range to determine the quality of the printed image to be judged.
According to U.S. Pat. No. 5,384,859, the process for quality control of an image, comprises: defining within a printed image to be inspected a plurality of individual image elements, each element encompassing an image pattern; storing a master printed image incorporating said plurality of individual image elements, the stored master image including for each image element a stored nominal pattern value; storing an acceptable tolerance range for each image element nominal pattern value to provide minimum and maximum allowable pattern values for each said image element; measuring individual image elements of a printed image to obtain individual measured pattern values; and comparing said individual measured pattern values with corresponding allowable maximum and minimum stored pattern values to determine errors in said printed image.
When producing certain types of valuable prints, such as securities, banknotes, stamps, etc., the images are printed using various printing techniques, such as offset, intaglio, etc. These various types of printing techniques constitute as many printing phases. In a normal printing process, the paper firstly passes through a printing system for the first phase and a first drawing is printed, and then the paper passes through a second printing system for the second printing phase enabling a second drawing to be printed on the paper. In this case, apart from the problem of printing quality, there is also the problem of printing the drawings of the different phases into a proper relative register. The reason for this is that deviations may exist between two images printed in this way in the case of drawings which are printed in different phases. These deviations, which may amount to a few pixels, may be either in the direction of movement of the paper or in a direction perpendicular. In this case, it is hardly possible to extract a model which represents the desired printing quality by using the techniques mentioned above since greatly varying values may be associated with the same pixel owing to misalignment or defective registration between the printing phases.
In this case, it has been proposed to construct a model for each printing phase. To do this, sets of sheets printed only with one of each of the printing phases are included in the set of test sheets. Using a procedure similar to the procedure described above, a model is constructed for each printing phase. During the phase of preparing these models, the operator identifies the portions of the image which comprise only or essentially only a single printing phase.
In production, firstly the relative misalignments between the printing phases are measured by using the pixels identified during the preparation of the models.
Next, the models are combined, taking into account the way in which the various phases are successively printed on the sheets in order to obtain a single reference model whose disposition corresponds to the disposition of the drawings in the images to be checked. Next, each image is compared with the model thus produced. This known procedure is complicated and particularly expensive for the printer, since for each production batch it is necessary to print as many sets of sheets representative of the desired printing quality as there are printing phases.
Other known methods are disclosed for example in EP 0 730 959 (corresponding to U.S. Pat. No. 5,778,088), EP 0 734 863 and EP 0 985 531 (corresponding to U.S. Pat. No. 6,665,424). EP 0 730 959 discloses a procedure for producing a reference model by electronic means, intended to be used for automatically checking the printing quality of an image on paper, especially for paper securities, said image being composed of drawings printed in at least two separate printing phases, which procedure comprises the following steps:
a. a set of images (test sheets), which are completely printed by means and procedures used for long print runs, is prepared;
b. said images are arranged so that the drawings of said images printed in a first phase are in register;
c. while the images remain in register they are recorded and the densitometric values of pixels constituting said images are stored in a memory;
d. the minimum value obtained from all the images of the set for each pixel location of the images of the set is associated with each pixel of a first printing phase model and the model of the drawing printed in said first printing phase is thus formed;
e. thereafter, the images are arranged so that the drawings printed in another printing phase are put into register and steps c. and d. are repeated for all the drawings printed in the separate printing phases; and
f. the models thus obtained are recombined in order to form the reference model of the final produced image on paper to be checked, each individual test sheet undergoes the same type and number of printing phases as the final produced image on paper to be checked.
Then the quality control of printed sheets is carried out by comparison of the obtained sheets with the reference model for example by comparing each pixel value obtained from a printed sheet with the corresponding pixel value of a reference image. If the pixel values obtained are within predetermined ranges of pixel values of the reference image, the controlled sheet is accepted, if not the sheet is rejected.
Another example of a control method is disclosed in EP 0 734 863. In this publication, the control process involves passing sheets of paper carrying printed images in front of a camera which inspects them and passes the captured image to a unit which is capable of measuring the misalignment. The measured value of misalignment is sent to a memory which contains all the pixel models which have acceptable misalignment and the model matching the measured value most closely is chosen and transmitted to a comparator. The comparator compares subsequent images captured by the camera with the selected pixel model thus establishing an automatic image control.
A further example of a process for producing by electronic means a model for automatically inspecting the print quality on deformable objects is given in EP 0 985 531. The model is firstly produced by capturing with an electronic camera (CCD for example) the images of a set of sheets whose print quality is regarded as acceptable; the images are stored so as to produce a first reference image, together with the relevant densitometric tolerance limits. This reference image is thereafter divided into a multitude of sub-images by superimposing a grid with very small mesh cells. During inspection, the distances between the nodes of the grid are measured on the image to be inspected: this therefore produces an elastic modification of the model, which is such as to make the distances between the nodes the same as in the image to be inspected. The image to be inspected is thus verified with respect to the modified reference (model) by using any of the standard inspection techniques.
Yet another example of a method and apparatus for controlling printing quality of banknotes during their production is disclosed in European patent application EP 0 582 548 A1. This document discloses an image processing apparatus including a data acquisition stage for acquiring data representative of several spatially separated regions of a sample. The apparatus also includes data storage means for storing reference data corresponding to the sample data, namely an image of a reference sheet. The two sets of data are compiled and analysed to determine if the sample is shifted from a nominal position.
Another type of control that is carried out on printed matter such as banknotes is described in U.S. Pat. No. 3,412,993: after printing, banknotes are inspected photoelectrically to determine whether the printing is correctly centred upon the note. An operator initially marks any imperfectly coloured banknotes whilst still in a sheet and after subsequent cutting, stacking and counting, the individual banknotes pass by means of rollers or a conveyer belt in front of two detecting systems each inspecting both sides of the banknote. A first detecting system senses the previously applied colour mark either optically or by the magnetic or electric properties of the marking ink and a second, photoelectric, system monitors the centring of the printing. Imperfect banknotes are discarded prior to priming with serial numbers or if already printed are replaced by perfect ones. Further counting takes place before final packaging. Correct centring is determined by measuring the width of the plain border surrounding the printing at two specific points along one edge and at one point along an adjacent edge. In one arrangement the banknote must be correctly aligned on the conveyor or rollers with respect to the detecting apparatus and is arranged with its longer edge transverse to the direction of motion. Two light beams are directed onto the conveyer at spaced points across the path of the note and with associated electronic multipliers receiving reflected light time the passage of the border along the longer edge as it goes by and so provide a measure of its width at the two points. The width of the border along the adjacent edge, which extends parallel to the direction of motion, is measured by an oscillating photo-electric device scanning the border and whose readings are taken at a certain time after the leading edge of the bank-note has been detected. The electrical signals representing the widths are compared with standard electrical signals to determine whether the note is acceptable.
A method used in the art to compare printed images with a reference image is called pattern matching. In this method, one determines a reference pattern in a reference image, then one looks for said predetermined reference pattern in a sample image of the print being inspected, whereby all possible position variations of the reference pattern within a search area of the inspected print are compared for a match. This implies both that (i) the reference pattern must be sufficiently unique to be robustly identified in the sample image and that (ii) the search area must be sufficiently larger than the reference pattern to be able to find all expected position variations of the reference pattern. An example of a prior art publication related to this field is “Digital Image Processing”, Gonzalez/Woods, Addison Wesley, page 583 (ISBN 0-201-50803-6).
More specifically, to carry out pattern matching, known methods can be used. This process involves mainly two phases: an off-line learning phase in which the template is processed, and a matching phase that can be executed in real time. The learning phase of pattern matching involves analysing the template image to find features that can be exploited for efficient matching performance. The matching phase uses the information from the learning phase to eliminate as much unnecessary calculation as possible.
Typically, the matching algorithm that can be used depends on whether the user has specified shift-invariant matching (finding the template at any location in the search image) or rotation-invariant matching (finding the template at any location AND rotation in the search image). Both are two-pass processes.
Shift-Invariant Matching
The first pass is a correlation that uses only the pseudo-randomly sampled pixels from the template image. The results of the stability analysis are used to determine how many positions in the search image can be skipped without missing any important features. For example, if all the sub-sampled pixels were found to be stable in a 3×3 neighborhood, the matching algorithm can skip two out of three correlations in each row and column while still guaranteeing that a match will be detected. This reduces the number of calculations required by a factor of 9. The first pass produces a number of candidate matches with rough position information.
The second pass only operates on the candidates identified in the first pass. The edge detection results of the learning phase are used to fine-tune the location of each match, and a score is produced for each based on the correlation result at that location. A user-provided score threshold determines which candidates are returned as matches.
Rotation-Invariant Matching
The first pass uses the circular intensity profile from the learning phase to search for shifted versions of that profile throughout the image. The user can input an allowable rotation range (in degrees) to reduce the number of calculations required in this pass. Several candidate matches are identified in this pass.
The second pass uses the pseudo-randomly sampled pixels to perform a correlation with all the candidates. A score is produced for each candidate to determine whether it should be classified as a match or not.
The choice of the template to be matched can have a great impact on the speed and accuracy of the pattern matching algorithm. There are some general observations:                The template should be asymmetric enough so that it can be uniquely identified at a certain orientation.        Complex templates will take longer to match than very simple ones. However, it must be ensured ensure that the template includes enough detail to uniquely identify and precisely localize the pattern.        The template should contain enough detail to fix its spatial position in the image. To do this, it needs to contain both vertical and horizontal features.        If one tries to locate a simple feature such as a dot or hole in a board, the template should be large enough to contain background information that distinguishes that particular feature from similar ones in the image.        
US patent application No. 2003/0194136 A1 discloses an example of a known pattern matching technique and an image processing device to carry out the said technique. This device is adapted to detect and extract a pattern which resembles a specified pattern within image data which is obtained from a sample printed document and to calculate the degree of resemblance between the extracted pattern and a reference pattern which was established beforehand. The disclosed device is in particular intended to be used is a colour-copy machine in order to detect paper money when someone attempts to copy it and prevent the copying process from proceeding to completion. According to this application, the whole surface of the sample printed document is accessible to the image acquisition device. According to US 2003/0194136 A1, detection of a specified pattern in the image data is carried out using masks of specified sizes and checking areas of the image for patterns which are possible matches with each specified pattern which is to be detected, e.g. a mark, figure, etc. If a possible candidate is detected, a reference position of the pattern is specified and the data is transmitted for further processing to extract the specified pattern and match this pattern with a reference pattern defined beforehand.
The solution described in US 2003/0194136 A1 might be adapted to application environments where the whole surface of the sample printed documents to be tested is available for inspection. Such solution is however inadequate for applications where only a limited portion of the surface of the printed documents is available for detection.
A similar approach and device is discussed in European patent application EP 0 382 549 A2. In this case also, the whole surface of the sample printed document (e.g. a banknote) is available for inspection. This solution is accordingly also unsuited for applications where only a limited portion of the surface of the printed documents is available for inspection.
Another field linked to the production of banknotes and similar products involves the counting of the products. Typically, in the field of banknotes, one produces a certain number of individual notes which are stacked in the form of bundles and packs after cutting of sheets into the individual notes, and it is important to count the notes present in each pile to control that the each bundle and pack comprises a predetermined number of notes.
A well-known counting device is disclosed in EP 0 737 936, the content of which is incorporated by reference in the present application. This device includes a counting disc for counting sheet-like substrates arranged in a stack, such as stacks of sheets or notes. More specifically, the counting disc comprises circumferential sections arranged on its border, and each circumferential section has a suction hollow in which suction openings located one behind the other are arranged. Upon rotation of the counting disc, said suction openings are connected intermittently to a suction-air source, with the result that the corners of a stack (for example a stack of banknotes), one after the other, are subjected to suction, deformed, separated from the rest of the substrates and, by virtue of a pneumatic counting pulse being produced, counted. The suction air is supplied to the suction openings via a duct whose section, which opens into the suction openings, is directed perpendicularly with respect to the plane of the counting disc. In the end, the number of pulses produced corresponds to the number of sheet-like substrates counted.
An evolution of this counting device is disclosed in International application WO 01/14111, the content of which is incorporated by reference in the present application. In this evolution, in addition to counting the piled substrates with a rotating disc based on the technology of EP 0 737 936, said disc comprises additional means to determine the location of printed patterns on the counted substrates to carry out a check comparable to the one disclosed in U.S. Pat. No. 3,412,993 cited above, i.e. for controlling the so-called print-to-cut register. The idea of International application WO 01/14111 is to provide means on the counting device that allow simultaneous control of the distance between an edge of the counted substrate and the printed images on said substrate, during the counting operation. Indeed, it is important to ensure that after the cutting operation a predetermined distance between the edges of the resulting substrate and the printed patterns thereon remains constant or at least within certain predetermined limits. The optical system disclosed in International application WO 01/14111 allows determination during the counting operation of said distance with respect to two edges of the substrate by a reflection measurement.
This combination of two operations (counting and distance measurement) is an advantage since it allows to reduce the time necessary to produce substrates meeting the desired quality requirements.
As mentioned above, it is sometimes important to measure the position of a print on paper relatively to the edges of the paper or between two different print processes. One example is the measurement of the position of the paper cut to the print on banknotes during the production process (the so-called print-to-cut or print-to-edge register). Normally, as in the prior art, one looks for an unique pattern in the acquired image to have an accurate measurement of the print position itself.
As already mentioned hereinabove, a condition to be met is the fact that the search area has to be larger than the search pattern to be found to cover all expected position variations.
Due to the limited available space in some machine environments it is sometimes only possible to have access to a very small viewing area of the print or paper sheet. Consequently, if one can see only a small part of the print and the variation in the normal printing process is larger than the size of the viewing area, the selected reference or search pattern can vary in its position in such a way that it may disappear from the viewing area. In this case it is not possible to use a common pattern matching method as discussed above (such as discussed in US 2003/0194136 A1 and EP 0 382 549) where the reference or search pattern is selected within a reference image, as taught by the prior art methods, and it is necessary to find a new approach. This problem is typically present in the device of International application WO 01/14111, but such problem also arises in other applications where only a small part of the print to be inspected can be viewed.